Use of Time Information in Models behind Adaptive 
System for Building Fluency in Mathematics 

Jin Rihak 

Masaryk University Brno 

thran@mail.muni.cz 


ABSTRACT 

In this work we introduce the system for adaptive practice 
of foundations of mathematics. Adaptivity of the system 
is primarily provided by selection of suitable tasks, which 
uses information from a domain model and a student model. 
The domain model does not use prerequisites but works 
with splitting skills to more concrete sub-skills. The student 
model builds on variation of Elo rating system which provide 
good accuracy and easy application in online system. The 
main feature of the student model is use of response times 
which can carry useful information about mastery. 

1. INTRODUCTION 

Our aim is to develop a practice system focused on basic 
mathematics which uses concepts of Computerized adaptive 
practice [8], i.e. to provide children with tasks that are most 
useful to them. We focus especially on detecting mastery 
and fluency using both correctness and timing information 
about children’s responses. 

Mathematics is usually associated with procedural knowl- 
edge. However, for achieving mastery of advanced topics it 
is necessary to solve some basic mathematical tasks at the 
level of fluency and automaticity. Good example of this is 
multiplication of small numbers which starts as procedural 
knowledge (child knows that 3-5 is 5 + 5 + 5 and is able 
to complete calculation) but ends as declarative knowledge 
(child knows 3 ■ 5 is 15 without further thoughts) [15]. In 
both cases child gives correct response with high probabil- 
ity and the system is not able to distinguish between these 
scenarios based only on the correctness of the answer. Thus 
we want incorporate into our student model the information 
about response time, which is necessary to detect mastery, 
the state when the child is correct and fast. 

Because our goal is to lead a child to automaticity we want 
to analyse strengths and weaknesses of the child at the level 
of individual items. Thus we need to track child’s skills in 
great detail and we treat every item in the system indepen- 


dently. Also the fact that various graphical representations 
of the same task influence difficulty of the item, highlight 
need to track their difficulty individually. To estimate cor- 
rectly difficulties of the items requires a lot of expertise, it 
is time consuming and is not always reliable. Therefore we 
do not want to make any assumptions about difficulties of 
the items and we rather use model which can estimate the 
difficulty of the solving data from the system. As a con- 
sequence we will be able to easily analyse which items are 
more difficult and why. 

Proposed system is called MatMat and is currently available 
online in beta version at matmat . cz for all children (the sys- 
tem is so far implemented only in Czech) and it is free to 
use. The goal of the system is to provide adaptive practice of 
arithmetic operations which guide children from basic work 
with numbers (e.g. counting objects) to mastery of basic 
mathematical operations. 

In contrast with complex intelligent systems for learning 
mathematics as Carnegie Learning’s Tutors [14, 9] or AS- 
SISTments [4] we focus only on small part of learning math- 
ematics and we work only with atomic tasks. Therefore the 
system does not work with explanations of curriculum or 
hints and focuses on adaptive selection of tasks and appro- 
priate feedback. Between related systems belongs Dybuster 
Calcularis [6] which works with basic math especially in con- 
text of dyscalculia; Math Garden [8] which has similar focus, 
works with similar student model and also incorporates time 
information; or FASTT Math [3] which also focus on build- 
ing computational fluency. 

2. MODELS 

In this section we describe working draft of the domain 
model, which describes how is the content of the system 
organized, and the student model, which is built on the do- 
main model and provides information about children who 
interact with the system. We have several requirements for 
the design of our models. We are in the situation when 
we use models in online environment and we rely more on 
collected data instead of expertise or other outside informa- 
tion. Hence we require models which can work on the fly 
and can quickly adapt to new data in the system. The goal 
of the student model is to provide estimation of child’s abil- 
ities which are used for creation of feedback and selection of 
suitable tasks to practice. 
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2.1 Domain Model 

Mathematics is very complex domain full of diverse compo- 
nents and relationships. Even in our very simplified case, 
when we considered only basics, situation can still be rela- 
tively complicated. One way how to build a domain model 
for mathematics is based on Knowledge space theory [1], 
This approach splits the curriculum to skills a defines re- 
lations of prerequisites between them. This oriented graph 
can then be treated as dynamic Bayesian network [7]. 


We used different approach which allows us to capture infor- 
mation about very specific abilities, e.g. how good is child in 
multiplication of 5 and 7. The relations between such con- 
crete skills, are not always prerequisites, e.g. the abilities to 
compute 5 • 7 and 5 • 9 are not one prerequisite to another 
but they are clearly dependent. We organized the skills into 
the tree structure (Figure 1) where every node corresponds 
to skill and its successors to more concrete sub-skills. Simi- 
larity of skills then can be expressed as level of the nearest 
common ancestor. Denote the fact that a skill d is ancestor 
of a skill c as d > c. 


level 1 j level 2 


level 3 


level 4 


math 


numbers 

si 

1 - 9 


1 

, , . . i A\ 




addition k v T 10-20 A ( j 2 


substruction 

£ x 

> 20 


division 

l.- I 

f 'J 

3x5 

multiplication 

\ 

[l-10]x[l-10] 


7x8 


[ll-20]x[l-10] 


Figure 1: The tree structure of the skills 


2.2 Student Model 

Rather then the discrete representation of ability (known 
or unknown) we used the continues representation, which is 
more suitable for our situation when we need to track abil- 
ities also for relatively general skills. The relation between 
these abilities and expected probability of correct answer is 
defined by a logistic function. 

For the skill from s and the child c model estimates the 
value v ac which represents difference of ability relative to 
parent skill. Overall value of ability is then 9 SC = '}Z s<s Vsc- 
This approach allows to capture relations between leafskills. 
Information obtained from observation about one ability can 
be naturally propagated to other related abilities. This is 
especially important for new children in the system with 
small number of responses (relatively to large number of 
abilities) . The model also estimates the difficulties /3i of the 
items i, which can be interpreted as a required ability to have 
50% chance of solving item correctly. Expected response is 
then e ci = 1+e fi\-o sc • 


To estimate abilities and difficulties we used a model based 
on Elo rating system [2] and PFA [12] which is inspired by 
models which have been successfully used in other projects 
[8, 11]. The main idea is to update all related abilities and 
item difficulty based on unexpectedness of response after 
every answer. To empathize the fact that the correct answer 
(even repetitive) does not mean mastery we need to take 
into account the response time td . This can be achieved 
by extension of discrete response r c » (correct or incorrect) 
to continuous one where values between 0 and 1 mean the 
correct answer but with longer time than the targeted time 
n. Example of this extension is decay of the response value 
exponentially relatively to the ratio of t c % and n (Figure 2). 


The root of the tree is a global skill which represents over- 
all knowledge of mathematics. Under that are skills which 
correspond to basic units in system (level-2) — numbers, 
addition, subtraction, multiplication and division. In level- 
3 are sub-skills which represent concepts (inspired by [6]) 
within parent skill, e.g. under ‘numbers’ skill are ‘numbers 
in range from 1 to 9’, ‘numbers in range from 10 to 20’, ‘num- 
bers greater then 20’, . . . ; or under ‘addition’ are ‘addition 
in range from 1 to 9 (without bridging to 10)’, ‘addition 
in range from 10 to 20 with bridging to 10’, ... And finally 
level-4 skills correspond to the tasks for which mastery on 
the level of declarative knowledge is expected. Example of 
these are skills that correspond to numbers (1, 2, 3, ...), 
simple addition tasks (1 + 2, 5 + 7) or multiplication of num- 
bers smaller than 10 (3 • 5, 7 • 8). There are no level-4 skills 
for more complicated task (e.g. 11 • 13) for which procedural 
knowledge is more involved. The items representing these 
tasks belong typically to more general level-3 skills. 

In current model every item in the system is mapped to 
exactly one skill (typically a leaf skill). So under a skill are 
multiple items. In case of the more general level-3 skills it 
can be tens or hundreds. In case of the level-4 skills there 
are from 2 to 10 items which are various forms of the task 
(5 + 7 and 7 + 5) and different graphical representations of 
task (numbers, objects, number line . . . ). 



Figure 2: The response value for the correct answers 

After the answer, all abilities 9 SC belonging to ancestors’ 
skills s are updated. Updates of abilities are performed se- 
quentially from the root of the skill tree. If the answer is the 
first answer of the child to the item, the difficulty of item /3i 
is also updated. 

Pi = Pi + T— (eci - r ci ), 

1 + p ■ rii 

9sc — 9sc + 7s ' ^ r (r ci eci). 

The parameters a and /3 define shape of the decay function 
[13] which prevents excessive influence of recent responses. 
The decay function takes argument n, - number of previous 
updates of that difficulty. Parameter K r corresponds to PFA 
updates and depends on correctness of answer. Parameter 
7 s G [0, 1] tells how much response to the item testifies about 
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a skill and consequently, how much information obtained 
from response is propagated to sub-skills. Reasonable values 
of 7 S are near 1 for the most concrete skills and near 0 for 
the global skill. 

2.3 Item Selection 

The selection of an appropriate item that suits the ability 
of a child is a key feature of the system and has to balance 
several aspects. The system should not select the same or 
similar item in a short time, it should select diverse items 
for better exploration of child’s abilities and, foremost, the 
system should select items with appropriate difficulty - not 
already mastered (high probability of success) and not too 
difficult (small probability of success). Currently used al- 
gorithm is very similar to the one described in [11], Only 
difference is in bringing into account also similarity of items 
(e.g. 5 + 7 is similar with 7 + 5). 

3. FUTURE WORK 

Most of adaptive educational systems currently work only 
with correctness of responses. Our goal is to find out if 
this classical approach can be robustly extended by taking 
into account timing information and if this extension can be 
useful in building fluency in the basic mathematical tasks. 
To target this questions we proposed the system described 
in this work. This system is still in testing phase but the 
first analysis of 28 thousand collected answers, show that the 
ability and difficulty values estimated by the student model 
make intuitive sense, the system can adapt quickly and the 
item selection algorithm works reasonably. However, there 
is a lot of space for improvement. 

The domain model can be enriched with prerequisites which 
can be useful for both ability estimation and for item selec- 
tion. The current choice of the skills used in the domain 
model should be reviewed by a domain expert or compared 
with automatic methods which use collected data [10]. The 
proposed student model is incorporating response time but 
current approach is quite simplified and explicitly does not 
distinguish between accuracy and speed, which can be mod- 
eled separately. Also it is not clear how to set, or rather au- 
tomatically estimate, targeted response times n. Next char- 
acteristic of the model is propagation of information about 
abilities across all skills, which is useful in first phases but 
later can be undesirable. The propagation is closely con- 
nected to parameters 7 S and their influence to the model 
behaviour should be investigated. 

To evaluate our approach the proposed models will be com- 
pared to alternative models (e.g. Bayesian network model 

[7] which works with prerequisites) or simpler versions of Elo 
model (e.g. model which uses only one global skill and inde- 
pendent local skills [11])- The comparison of the models can 
be done offline with respect to the quality of predictions or 
online by comparison of an improvement rate or behaviour 
of children groups using different models and item selection 
strategies. These comparisons should bring some light an 
whether the proposed methods are useful. 
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